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ABSTRACT 

An evaluation was done of the New York City Public 
Schools 1 Project Porvenir. During the 1990-91 school year, Project 
Porvenir operated at 4 elementary schools and served 501 
Spanish-speaking students in grades 4, 5, and 6, of whom 85 percent 
were eligible for the free lunch program. The students were of 
limited English proficiency and were served with bilingual general 
and special education classes aimed to develop skills in English as a 
Second Language and native language arts as well as work skills 
through career education Project Porvenir also aimed to increase 
parent awareness of career opportunities for their children. The 
evaluation was conducted using student and director questionnaires, 
test scores, and interviews. Analysis found that cooperation between 
regular school staff and project staff was a strength of the program 
and that the regular school staff assumed increased responsibility 
for implementing bilingual career awareness in their classes. The 
project was also found to have used effective strategies in 
mainst reaming students and developed functional bilingual career 
resources center in the four project schools. The following 
recommendations are highlighted: (1) increased data collection; (2) 
modification of objectives; and (3) examination of reasons why the 
percentage of students meeting attendance objectives was so low. 
(JB) 
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FOREWORD 



The body of this report is preceded by an Extract which presents an overview of salient points of 
the project: funding cycle; enrollment figures; background of students served; admission criteria; and 
programming features, strengths, and limitations, including the outcome of all objectives* The extract also 
presents the conclusions drawn by the Office of Research, Evaluation, and Assessment (OREA) about the 
program and its recommendations for program improvement. 

The extract is followed by the body of the report, titled Pro gram Assessment . This includes such 
information as staffing, program implementation, and outcome and implementation objectives. 
Instructional objectives are presented first, followed by noninstructional objectives. The report then 
addresses those aspects of programming mandated by Title VII regulations that do not have specifically 
stated objectives. This may be information on attendance and dropout rate, grade retention, 
mainstreaming, referrals out of the program to meet special needs of the students, and withdrawals. A 
case history concludes the report. 

Data for this profile were collected and analyzed using a variety of procedures, which are 
described in Appendix A following the text. 



Office of Research, Evaluation, and Assessment 
New York City Public Schools 

E.S.E.A. Title VII Evaluation Profile 

Project Porvenir 

Grant Number: T003A80165 
1990-91 

EXTRACT 

PROJECT DIRECTOR: Dr. Frances Segan FUNDING CYCLE: Year 3 of 3 

SITES 

School Community School District Grade Levels Enrollment* 



P.S. 25X 
P.S. 32X 
P.S. 65X 
I.S. 206X 



7 
10 
7 
10 



4, 5,6 
4, 5, 6 
4, 5,6 
5, 6 - 



90 
144 
140 
127 



*The project enrolled a total of 501 students (98 less than in the previous year, but more than the 284 
anticipated). 



STUDENT BACKGROUND 



Native Language Number of Students Countries of Origin Number of Students 

Spanish 501 United States 205 

Puerto Rico 116 

Dominican Republic 96 

Mexico 42 

Honduras 10 

Other countries 24 

Unknown 8 



Mean Years of Education in Native Country : 2.4; in the United States : 4.2 
Percentage of Students Eligible for Free Lunch Program : 85.0 



ADMISSION CRITERIA 

The project admitted students who had scored at or below the 40th percentile on the English Language 
Assessment Battery (LAB) and were thus classified as limited English proficient (LEP). The project also 
admitted students who, in addition to their LEP status, were certified for special education. 



PROGRAMMING 
Features 

Project Porvenir served Spanish-speaking LEP students in bilingual general and special education classes. 
It aimed to develop skills in English as a Second Language (E.S.L.) and Native Language Arts (N.L.A.) as well as 
work skills through career education. It also aimed to increase parent awareness of career opportunities for their 
children. 



9 

ERIC 



7 



Strengths and Limitations 



Cooperation between regular school staff and project staff was a strength of the program. The regular 
school staff assumed increased responsibility for implementing bilingual career awareness in their classes. The 
project used effective strategies in mainstreamiug students and developed functional bilingual career resource 
centers in the four project schools. 

CONCLUSIONS AND RECOMMENDATIONS 

Project Porvenir was fully implemented. It met its objectives for career education, attendance, staff 
development, and parental involvement. The project did not provide the information necessary for OREA to 
evaluate the objective for mainstreaming. Project Porvenir did not meet its objectives for E.S.L., N.L.A., and 
special education attendance. The project failed to realize the objective for curriculum development, although the 
materials were in draft form. 

The conclusions, based on the findings of this evaluation, lead to the following recommendations: 

• Supply OREA with all data necessary to evaluate objectives. 

• Modify objectives to make them more realistic and measurable. 

• Explore reasons why the percentage of students meeting the E.S.L., N.L.A., and special education 
attendance objectives was so low. 
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STAFFING 

Title VII Staff (Total 5) 



PROGRAM ASSESSMENT 



Title 

Project Director 
Resource Teacher 
Paraprofessional 
Paraprofessiooal 
Office Associate 



Degree Language Competencies 



Ph.D. 
M.A. 



Spanish 
Spani: l \ 
Spanish 
Spanish 
Spanish 



Other Staff Working With Project Students (Total 19) 



Title VII Funding 

Full time 
Full time 
Full time 
Full time 
Full time 



Degree Certification 

B.A. 7 Common Branches 5 

M.A. 10 Bilingual Common Branches 7 

M.S. 2 Bilingual Special Education 7 



Language Competencies 
Spanish 19 



Teaching/Communicative 
Proficiencies (TPfCP)* 

TP 19 



All staff were certified in areas they served except for five teachers certified in common branches who 
taught bilingual classes. 



IMPLEMENTATION AND OUTCOMES (Objectives prefaced by •) 

English as a Second Language (E.S.L.) 

E.S.L. was taught five periods a week for all except three students. These three received E.S.L. 
instruction only three days a week as a supplement to the classroom teacher's instruction. 

Teachers used a holistic, interdisciplinary approach that incorporated the use of the four E.S.L. language 
skills (listening, speaking, reading, and writing) through role-playing, the use of films and tapes, and interviews of 
guest speakers. Paraprofessionals provided individual or small group tutoring. 

The project staff offered demonstration lessons which included the use of E.S.L. picture files for job 
clusters, student-made books, hands-on displays, and students' journal entries. Students participated in a career 
fashion show to dramatize various occupations. The students cut pictures of clothing from magazines and 
newspapers and used them as models for their fashion show. The project staff made suggestions to teachers for 
follow-up E.S.L. activities that teachers could also use in the content areas. For example, the student-caterer 
needed to know how to read, use mathematics, understand the value of a proper diet and the need for health 
precautions, and to know about a variety of foods from different countries. Participants in the fashion show wrote 
thank-you notes to guests. 

As a follow-up activity, they wrote letters to guests thanking them for coming to the show. The teachers 
taught the students how to keep logs and write reports about special events and how to interview guest speakers 
during International Career Days. The students recorded their stories and logged daily events in their individual 
journals. 



Teaching Proficiency (TP): Competent to teach in this language. 
Communicative Proficiency (CP): Conversational capability only. 



All levels of instruction used the Troll Career Awareness kits, the Read-Along tapes, the National 
Textbooks' Everyday English. Across the Curriculum, and Survival Skills for E. S. L. instruction. They also used 
the Oxford Spanish/ English Picture Dictionaries and tapes. 

• By June 1991, 75 percent of the student participants will significantly increase their performance on 
the English LAB. 

Evaluation Instrument: Language Assessment Battery (LAB)* 

Pretest: March 1990; posttest: March 1991. 

Number of students for whom pre- and posttest data were reported: 272 
Percent of students showing gain: 4L2 
Project did not meet objective. 



Native Language Arts (N.L.A.) 

Estimated percentage of students lacking literacy skills in native language: 25-30 

The teachers conducted activities during which students interviewed career role models who visited their 
classrooms. They then wrote letters thanking these visitors. Students also wrote reports in Spanish about career 
opportunities and made daily entries in their class journals. 

Classes used the Hampton-Brown Spaulding Puedo Ser series, the Santillana Career Series in Spanish, 
Oxford Picture Dictionaries, and tapes in Spanish and English. 

• By June 1991, 75 percent of the student participants will demonstrate a significant increase in their 
performance on the Spanish Language Assessment Battery (LAB). 

Evaluation Instrument: Language Assessment Battery (Spanish) 

Pretest: March, 1990; posttest: March 1991 

Number of students for whom pre- and posttest data were reported: 140 
Percent of students showing gain: 38.6 
Project did not meet objective. 



* OREA used a gap reduction design to evaluate the effect of supplementary instruction on project students* 
performance on the LAB. Since all LEP students in New York City are entitled to such instruction, no valid 
comparison group exists among these students, and OREA used instead the group on which the LAB was 
normed. Test scores are reported in Normal Curve Equivalents (N.C.E.s), which are normalized standard scores 
with a mean of 50 and a standard deviation of 21.06. It is assumed that the norm group has a zero gain in 
N.C.E.s in the absence of supplementary instruction and that participating students* gains are attributable to 
project services. 
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Career Education 



Participating students interviewed speakers, i.e. a parent who was a chef, an Hispanic caterer, and the 
dean of students at Lehman College. 

• By June 1991, 75 percent of the LEP students in bilingual general education and bilingual special 
education participating in the project will demonstrate awareness and competencies related to career 
education, articulation with junior high school, and bilingual high school options as measured by 
Project Porvenir-developed tests. 

Evaluation Instrument: Project-developed tests. 

Data were available for 422 students. Out of this number, 394 (93.4 percent) passed all tests. 

Project met objective. 



Attendance 

• By June 1991, 75 percent of the limited English proficiency (LEP) students in bilingual general 
education classes participating in the project will demonstrate comparable attendance rates to those 
of mainstream students as measured by official attendance records. 

Evaluation Indicator: Attendance records. 

School Project Students' Rate Mainstream Students' Rate 

P.S. 25 95.2 89.6 

P.S. 32 93.6 84.5 

P.S. 65 88.8 85.2 

I.S. 206 93.3 86.1 

Attendance of participating students was higher than that of mainstream students at all schools. 

Project met objective. 

• 

• By June 1991, 75 percent of the limited English proficiency (LEP) students in bilingual special 
education classes participating in the project will demonstrate significant improvement in their 
attendance as measured by official attendance records. 

Evaluation Indicator: Attendance records. 

Data were available for 27 bilingual special education students. Of these, only eight (30 percent) 
improved their attendance from the previous year. 

Project did not meet objective. 



Mainstreaming 

• By June 1991, 75 percent of the LEP student participants in bilingual special education will 
demonstrate improved skills in academic and non-academic mainstreaming, as well as social 
interaction with peers, as indicated on their individual educational programs (I.E.P.s) and 
documented by field-site teachers, using a Project Porvenir checklist. 

Evaluation Indicator: I.E.P.s and project teacher checklist. 



9 

ERLC 



5 ii 



To mainstream bilingual special education with bilingual general education students, the project arranged 
field trips for students to visit Pregones Theater, Fashion Industries High Schools, and Loew's Summit Hotel 
Bilingual general and special education and monolingual English general and special education students attended 
the International Career Days together. 

Project Porvenir, however, did not provide the data necessary to evaluate the academic mainstreaming 
portion of this objective. 

OREA could not evaluate objective* 



Staff Development 

The project director reported that she had made materials, such as cooperative learning tapes, available 
to all staff for training purposes. 

The project and the Multifunctional Resource Center at Hunter College collaborated in presenting two 
citywide workshops. Project Porvenir conducted four on-site after-school workshops. All staff participated. 

• By June 1991, 75 percent of participating teachers will have demonstrated mastery of ten teaching 
skills developed in training workshops or demonstration lessons, as documented by project director's 
records. 

Evaluation Indicator: Project director's records. 

The project director indicated that all participating teachers had developed or reinforced at least ten 
teaching skills. 

Project met objective. 



Curriculum Development 

• By August 1991, a final edition of curriculum and manuals for staff and parents will be completed 
and ready for dissemination. 

Evaluation Indicator: Completed curriculum guide and manuals. 

The program submitted an outline and draft of a projected curriculum guide but bad not yet finalized it. 

Project did not meet objective. 



Parent Involvement 

• By June 1991, involvement of parents of participating LEP students will have increased over the 
previous year based on such activities as parent workshops, individual conferences, I.E.P. conferences, 
and teacher contacts, as shown by program records. 

Evaluation Indicator: Program records. 
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The project offered a parent workshop, three parent conferences, and two Awareness Workshops, one of 
which featured a presentation by the president of the Federal Hispanic Employees Progrc .1 Council. The Parent 
Advisory Committee met once. Parents participated in trips and other activities and volunteered to serve as 
presenters and guest speakers. 

The project developed draft copies, in English and Spanish, of a booklet on mainstreaming and an 
informational packet describing the project, its major program goals, and its activities to be disseminated to 
parents. 

According to the project director, parental involvement had increased over the previous year. 
Project met objective. 



CASE HISTORIES 

E. was a special education student in the fifth grade who had only been in this country for a few months 
and spoke only Spanish. Both of his parents worked in low paying jobs. E.'s previous education in his native 
country of Mexico had been limited. He was enrolled in Project Porvenir where one of the paraprofessionals 
established a very close relationship with him. E.'s attendance improved greatly and he seemed to look forward 
io coming to school, particularly after he started learning English. Subsequently, he played an active role in a 
career education fashion show. He wore a chef 's outfit and did an excellent job explaining, in Spanish, what his 
duties would be as a chef. He was very proud of his accomplishments in class and has almost reached grade level. 
He has also helped his parents learn English by conversing with them in that language. 

A. was a general education fourth grade student who was born in Puerto Rico and had been living in the 
United States for three years. He needed improvement in his reading and writing skills in both English and 
Spanish. He made good progress in Project Porvenir. He read and pronounced words from his Oxford picture 
dictionary with little difficulty. His self-confidence improved considerably; he was no longer shy. Recently, he 
participated in a career education program at the Pregones Theater. He has become very cooperative and his 
school attendance has improved greatly* 



STUDENTS WITH SPECIAL ACADEMIC NEEDS 



Number of Students 



Percentage 



Referred to special education: 
Referred to remedial programs: 
Referred to gifted and talented programs: 
Retained in grade: 



0 
0 
0 

22 



0 
0 
0 
4.4 
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APPENDIX A 



DATA COLLECTION AND ANALYSIS 

Student Data Form 

This one-page form is filled out by staff for each participating and mainstreamed student. OREA gathers 
data from this form on backgrounds, demographics, academic outcomes, attendance, referrals, and exit from the 
program. 

Project Director's Questionnaire 

The Project Director's Questionnaire includes questions on staff qualifications, program implementation, 
periods of instruction, and instructional materials and techniques. 

Project Director's Interview 

The interview gathers information on program and student or staff characteristics not supplied by the 
Project Director's Questionnaire. The interview also allows project staff to offer qualitative data or amplify 
responses to the Questionnaire. 

Citvwide Test Scores 

OREA retrieves scores centrally from the Language Assessment Battery (LAB) and other citywide tests. 
For evaluation purposes, these test scores are reported in Normal Curve Equivalents (N.C.E.s). N.C.E.s are 
normalized standard scores with a mean of SO and a standard deviation (s.d.) of 21.06. They constitute an equal- 
interval scale in which the distance is the same between any two adjacent scores. A gain of 5 N.C.E.s is the same 
whether it is at the lower or the higher end of the scale. N.C.E.S can be used in arithmetic computations to 
indicate group progress. (Percentile scales, although more familiar to many, are unsuitable for such computations 
since they are not equal-interval.) 

Likert-TVpe Surveys 

Likert-type surveys, in which respondents mark their opinions on a scale from one to five, are used in a 
variety of ways. They examine student attitudes (i.e., toward school and career, native language use, and native 
and mainstream cultures). They also assess staff aud parent attitude and reactions to workshops and other 
activities. 

Gap Reduction Evaluation Design 

OREA uses a gap reduction design for measuring changes in standardized tests. Since no appropriate 
non-project comparison group is available in New York City, where all students of limited English proficiency 
• (LEP) are entitled to receive supplementary services, OREA compares the progress of participating students with 
that of the group on which the test was normed. It is assumed that the norm group would show a zero gain in 
the absence of instruction, and gains made by project students could be attributed to project services. 
(See "Citywide Test Scores" above.) 

To test whether pre/pofcttest gains are greater than could be expected by chance alone, OREA uses a 
t-test. To determine whether a difference between two proportions (e.g., program and mainstream attendance 
rates) is greater than could be expected by chance, OREA uses a z-test. The level of significance is set at .05 for 
all tests. 
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